Augmented Music-Understanding Interfaces

One of my research goals is to enrich music listening experiences by deepening each person’s understanding of music. Music listening experiences depend on music-understanding abilities. Although the music-composing/performing abilities of musicians are often discussed, the music-understanding abilities of casual listeners have not been well discussed or studied. It is difficult, for example, to define music-understanding abilities and express the results of understanding. In addition, it is difficult to know how others understand music. Music listening is usually an individual experience, and it is impossible to directly observe how others understand elements in music. Similarly, it is common to not notice what one does not understand when listening to music. Because of this, even if listeners want to better understand music or want to improve their ability to understand music, methods to realize those wishes have not yet been established and will have to be discovered. I therefore propose a research approach “Augmented MusicUnderstanding Interfaces that facilitates deeper understanding of music by using automatic music-understanding technologies based on signal processing. First, visualization of music plays an important role in augmenting people’s understanding of music because understanding is deepened through seeing. By recognizing and visualizing elements in music, we let end users (listeners) without expertise understand the existence of elements and the relationships between elements. Second, “music touch-up” (small modifications) also helps each person’s understanding of musical audio signals because understanding is deepened through editing. Music touch-up is a term I coined [1] that means making small changes to elements in existing music. It lets users naturally observe why music is composed and arranged in a certain way while casually enjoying the content modification (i.e., personalization or customization).